Papers with Reward-Augmented Decoding
Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models generate text that is problematic or lacks a desired attribute. |
| Approach: | They propose a text generation procedure that uses a small unidirectional reward model to encourage a language model to generate text with certain properties. |
| Outcome: | The proposed procedure outperforms prior weighted decoding methods and matches state-of-the-art techniques that require additional training. |